1. Field of the Invention
This invention relates to the visual tracking of objects in captured digital images, and segmentation of images, and the uses of such tracking, including (but not limited to) the control of camera(s) to track object(s) as the object(s) and/or the camera(s) move.
2. Background of the Invention
Tracking objects, or targets, in a series of captured 2-D images (e.g. an image from a digital video camera, or from a digitised image produced from an analogue video camera) is a known technical field in its own right. Furthermore, many other fields use the ability to track a moving image as a given achievable thing. For example in the area of face recognition from surveillance cameras, many people simply assume that the person's face can be tracked from frame to frame and the relevant image data input to a recognition algorithm. However, automatically tracking targets in an image that move from frame to frame is not so easy. One known problem is deciding from frame to frame which pixels in each image frame are part of the object/target, sometimes termed foreground, and which are the background—i.e. “not target”.
Some approaches establish a boundary that separates the target from the background in each frame. Some then use that boundary to define a shape and track the centre of the shape.
However, objects/targets can change orientation with respect to the camera acquiring the images and/or the camera can move relative to the target. The target can get closer or further away, and it can change pose, or orientation, relative to the camera. So the target can look quite different as judged by its boundary in a captured image from time to time. It is therefore not desirable to have a fixed, single, predicted shape for the target boundary shape. Some people have tried using models that have boundaries that evolve over time, from one captured frame to a different captured frame, to enable the target to be recognised as such and tracked
Fast and reliable visual tracking is a prerequisite for a vast number of applications in computer vision. Though it has been the subject of intense effort over the last two decades, it remains a difficult problem for a number of reasons. In particular, when tracking previously unseen objects, many of the constraints that give reliability to other tracking systems—such as strong prior information about shape, appearance or motion—are unavailable. One technique that has shown considerable promise for its ability to perform tracking and segmentation within a unified framework is the use of an implicit contour, or level-set to represent the boundary of the target. As well as handling topological changes seamlessly, tracking using level-sets can be couched in a fairly standard probabilistic formulation, and hence can leverage the power of Bayesian methods.
One technical difficulty is finding in an image where the object is located. A second technical difficulty is dealing in changes in the apparent shape of an object caused by changes in the relative position and orientation of the object and the image-acquiring camera.
Solutions in the past include that discussed in U.S. Pat. No. 6,394,557 (Leroi) and U.S. Pat. No. 6,590,999 (Comaniciu). Also known is reference [4] (see list of references). The reader of this patent is hereby instructed to read those three documents. We feel this will assist in understanding the present invention. None of these three documents are quite what we think is needed.